[Additional Layer Store] Enable Additional Layer Store to perform registry authentication #2417

ktock · 2024-05-14T13:49:09Z

Implements openshift/enhancements#1600 (comment)

Currently, c/image doesn't share creds to Additional Layer Store so it requires additional configurations for registry authentication. And the current means of registry authentication for Additional Layer Store doesn't seem to meet requirements for the platform: openshift/enhancements#1600 (review)

This commit fixes that issue by allowing c/image directly sharing creds to Additional Layer Store. Additional Layer Store doesn't need to have its own logic to fetch registry creds but it can receive them from c/image.

Additional Layer Store needs to provide a helper binary that is executed from c/image. This helper binary is registered to c/image using registries.conf with the following field (store-helper can be any command name of the helper binary). It receives registry creds via stdin and Additional Layer Store can use that creds for registry authentication.

additional-layer-store-auth-helper = "store-helper"

An example draft implementation of the helper binary is store-helper of stargz-store: containerd/stargz-snapshotter#1674
This binary is executed by c/image and receives the registry creds from stdin and shares them to stargz-store via the unix socket of stargz-store. Then stargz-store uses these creds for registry authentication.

c/image passes DockerAuthConfig structures with keying them with the image reference.

{
  "image-reference": {
    "username": "username",
    "password": "password",
    "identitytoken": "identitytoken"
  }
}

mtrmac

Thanks!

This is plausible, assuming the system is single-purpose; otherwise all other registry operations, including manual skopeo commands, would also be sending credentials.

A possible alternative would be to capture the credentials in ImageSource, and provide a some kind of interface alongside TryReusingBlobOptions.SrcRef. That would only send “relevant” credentials to the helper, at the cost of some extra plumbing.

pkg/sysregistriesv2/system_registries_v2.go

docker/docker_image_src.go

mtrmac · 2024-05-14T14:09:23Z

docker/docker_image_src.go

+			Password      string `json:"password,omitempty"`
+			IdentityToken string `json:"identitytoken,omitempty"`
+		}{
+			physicalRef.ref.String(): {


Should this be .ref.Name()? Credentials are, for the most part, repo-scoped.

Admittedly the FUSE interface uses ref.String().

docker/docker_image_src.go

TomSweeneyRedHat · 2024-05-15T22:02:26Z

@ktock Skopeo test is failing and you need a rebase.

ktock · 2024-05-16T01:34:42Z

Rebased. And skopeo error seems to be resolved.

rhatdan · 2024-05-16T18:29:20Z

@mtrmac PTAL

pkg/sysregistriesv2/system_registries_v2.go

docker/docker_image_src.go

mtrmac

Thanks! This looks good overall.

I am still conceptually a bit unsure whether capturing all credentials ever read is a good idea — or whether we should be restricting it to those relevant to LookupAdditionalLayer, at the cost of having to make the special case visible to the c/image/copy top level. I don’t have a good idea of where this is likely to be deployed.

Other than that, the option and the data format just need a man page update.

@giuseppe PTAL as well.

docker/docker_image_src.go

mtrmac

Thanks!

Implementation LGTM, this is just missing man page updates. But I’d like to hear what @giuseppe thinks.

ktock · 2024-05-22T04:20:15Z

Added docs to docs/containers-registries.conf.5.md

docs/containers-registries.conf.5.md

ktock · 2024-05-27T15:03:45Z

Could we move this forward?

TomSweeneyRedHat · 2024-05-29T20:21:45Z

@giuseppe thoughts?

@ktock, looks like you need a rebase.

TomSweeneyRedHat · 2024-05-29T20:52:20Z

docs/containers-registries.conf.5.md

@@ -19,6 +19,12 @@ Container engines will use the `$HOME/.config/containers/registries.conf` if it
 `credential-helpers`
 : An array of default credential helpers used as external credential stores.  Note that "containers-auth.json" is a reserved value to use auth files as specified in containers-auth.json(5).  The credential helpers are set to `["containers-auth.json"]` if none are specified.

+`additional-layer-store-auth-helper`
+: A string of helper name. This enables to pass registry credentials to


Suggested change

: A string of helper name. This enables to pass registry credentials to

: A string containing the helper binary name. This enables passing registry credentials to an

TomSweeneyRedHat · 2024-05-29T20:53:04Z

docs/containers-registries.conf.5.md

+`additional-layer-store-auth-helper`
+: A string of helper name. This enables to pass registry credentials to
+  Additional Layer Store every time an image is read using the `docker://`
+  transport so that it can access private registries. See the later section for


Suggested change

transport so that it can access private registries. See the later section for

transport so that it can access private registries. See the 'Enabling Additional Layer Store to access to private registries' section below for

TomSweeneyRedHat · 2024-05-29T20:53:48Z

docs/containers-registries.conf.5.md

@@ -254,6 +260,30 @@ in order, and use the first one that exists.

 Note that a mirror is associated only with the current `[[registry]]` TOML table. If using the example above, pulling the image `registry.com/image:latest` will hence only reach out to `mirror.registry.com`, and the mirrors associated with `example.com/foo` will not be considered.

+### Enabling Additional Layer Store to access to private registries
+
+The `additional-layer-store-auth-helper` option enables to pass registry


Suggested change

The `additional-layer-store-auth-helper` option enables to pass registry

The `additional-layer-store-auth-helper` option enables passing registry

TomSweeneyRedHat · 2024-05-29T20:54:24Z

docs/containers-registries.conf.5.md

+### Enabling Additional Layer Store to access to private registries
+
+The `additional-layer-store-auth-helper` option enables to pass registry
+credentials to Additional Layer Store so that it can access private registries.


Suggested change

credentials to Additional Layer Store so that it can access private registries.

credentials to an Additional Layer Store so that it can access private registries.

However, here and above, you could use "the" instead of "an" if you prefer.

TomSweeneyRedHat · 2024-05-29T20:57:48Z

docs/containers-registries.conf.5.md

+The `additional-layer-store-auth-helper` option enables to pass registry
+credentials to Additional Layer Store so that it can access private registries.
+
+Additional Layer Store needs to provide a helper binary. This helper binary is


I think this is what you're getting at

Suggested change

Additional Layer Store needs to provide a helper binary. This helper binary is

When accessing a private registry via an Additional Layer Store, a helper binary needs to be provided. This helper binary is

TomSweeneyRedHat · 2024-05-29T20:58:13Z

docs/containers-registries.conf.5.md

+credentials to Additional Layer Store so that it can access private registries.
+
+Additional Layer Store needs to provide a helper binary. This helper binary is
+registered to `additional-layer-store-auth-helper` option. Every time an image


Suggested change

registered to `additional-layer-store-auth-helper` option. Every time an image

registered via the `additional-layer-store-auth-helper` option. Every time an image

TomSweeneyRedHat · 2024-05-29T20:59:26Z

docs/containers-registries.conf.5.md

+
+The format of `$image_reference` is `$repo{:$tag|@$digest}`.
+
+Additional Layer Store can use this for accessing to the registry.


Suggested change

Additional Layer Store can use this for accessing to the registry.

Additional Layer Stores can use this helper binary to access the private registry.

ktock · 2024-05-30T01:00:41Z

@TomSweeneyRedHat Thanks for the review. FIxed and rebased the patch.

ktock · 2024-06-05T02:24:22Z

@giuseppe PTAL

giuseppe

LGTM

Signed-off-by: Kohei Tokunaga <[email protected]>

mtrmac · 2024-06-05T13:30:42Z

Thanks again!

ktock mentioned this pull request May 14, 2024

[Additional Layer Store] Add authentication helper containerd/stargz-snapshotter#1674

Draft

mtrmac requested changes May 14, 2024

View reviewed changes

ktock force-pushed the storeauth branch from 6e51086 to 9edd5cd Compare May 15, 2024 02:05

ktock force-pushed the storeauth branch from 9edd5cd to e15e4f6 Compare May 16, 2024 01:11

mtrmac reviewed May 20, 2024

View reviewed changes

pkg/sysregistriesv2/system_registries_v2.go Outdated Show resolved Hide resolved

docker/docker_image_src.go Outdated Show resolved Hide resolved

mtrmac reviewed May 20, 2024

View reviewed changes

docker/docker_image_src.go Outdated Show resolved Hide resolved

ktock force-pushed the storeauth branch 2 times, most recently from 7cc7a9d to f5a3117 Compare May 21, 2024 00:18

mtrmac reviewed May 21, 2024

View reviewed changes

ktock force-pushed the storeauth branch 3 times, most recently from 002cfce to 55844d4 Compare May 22, 2024 01:43

mtrmac reviewed May 22, 2024

View reviewed changes

docs/containers-registries.conf.5.md Outdated Show resolved Hide resolved

docs/containers-registries.conf.5.md Outdated Show resolved Hide resolved

ktock force-pushed the storeauth branch from 55844d4 to 60696fc Compare May 23, 2024 01:46

TomSweeneyRedHat reviewed May 29, 2024

View reviewed changes

ktock force-pushed the storeauth branch 2 times, most recently from f93b7a9 to 9d45ea6 Compare May 30, 2024 00:36

ktock force-pushed the storeauth branch from 9d45ea6 to b9a9abf Compare June 3, 2024 15:28

giuseppe approved these changes Jun 5, 2024

View reviewed changes

mtrmac approved these changes Jun 5, 2024

View reviewed changes

ktock added 2 commits June 5, 2024 15:09

Support Additional Layer Store's authenticaiton helper

6ddd954

Signed-off-by: Kohei Tokunaga <[email protected]>

Add docs about additional-layer-store-auth-helper option

6c1598a

Signed-off-by: Kohei Tokunaga <[email protected]>

mtrmac force-pushed the storeauth branch from b9a9abf to 6c1598a Compare June 5, 2024 13:09

mtrmac merged commit 63ecb5f into containers:main Jun 5, 2024
9 checks passed

ktock mentioned this pull request Jun 15, 2024

OCPNODE-2205: Lazy image pull support openshift/enhancements#1600

Open

ktock deleted the storeauth branch June 18, 2024 04:36

ktock mentioned this pull request Jun 25, 2024

Registry-related configuration for Stargz Store containerd/stargz-snapshotter#1527

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Additional Layer Store] Enable Additional Layer Store to perform registry authentication #2417

[Additional Layer Store] Enable Additional Layer Store to perform registry authentication #2417

ktock commented May 14, 2024

mtrmac left a comment

mtrmac May 14, 2024

TomSweeneyRedHat commented May 15, 2024

ktock commented May 16, 2024

rhatdan commented May 16, 2024

mtrmac left a comment

mtrmac left a comment

ktock commented May 22, 2024

ktock commented May 27, 2024

TomSweeneyRedHat commented May 29, 2024

TomSweeneyRedHat May 29, 2024

TomSweeneyRedHat May 29, 2024

TomSweeneyRedHat May 29, 2024

TomSweeneyRedHat May 29, 2024

TomSweeneyRedHat May 29, 2024

TomSweeneyRedHat May 29, 2024

TomSweeneyRedHat May 29, 2024

TomSweeneyRedHat May 29, 2024

ktock commented May 30, 2024

ktock commented Jun 5, 2024

giuseppe left a comment

mtrmac commented Jun 5, 2024

	: A string of helper name. This enables to pass registry credentials to
	: A string containing the helper binary name. This enables passing registry credentials to an

	transport so that it can access private registries. See the later section for
	transport so that it can access private registries. See the 'Enabling Additional Layer Store to access to private registries' section below for

	The `additional-layer-store-auth-helper` option enables to pass registry
	The `additional-layer-store-auth-helper` option enables passing registry

	credentials to Additional Layer Store so that it can access private registries.
	credentials to an Additional Layer Store so that it can access private registries.

	Additional Layer Store needs to provide a helper binary. This helper binary is
	When accessing a private registry via an Additional Layer Store, a helper binary needs to be provided. This helper binary is

	registered to `additional-layer-store-auth-helper` option. Every time an image
	registered via the `additional-layer-store-auth-helper` option. Every time an image


		The format of `$image_reference` is `$repo{:$tag\|@$digest}`.

		Additional Layer Store can use this for accessing to the registry.

	Additional Layer Store can use this for accessing to the registry.
	Additional Layer Stores can use this helper binary to access the private registry.

[Additional Layer Store] Enable Additional Layer Store to perform registry authentication #2417

[Additional Layer Store] Enable Additional Layer Store to perform registry authentication #2417

Conversation

ktock commented May 14, 2024

mtrmac left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TomSweeneyRedHat commented May 15, 2024

ktock commented May 16, 2024

rhatdan commented May 16, 2024

mtrmac left a comment

Choose a reason for hiding this comment

mtrmac left a comment

Choose a reason for hiding this comment

ktock commented May 22, 2024

ktock commented May 27, 2024

TomSweeneyRedHat commented May 29, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ktock commented May 30, 2024

ktock commented Jun 5, 2024

giuseppe left a comment

Choose a reason for hiding this comment

mtrmac commented Jun 5, 2024